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Abstract 

This paper considers a class of strategic scenarios in which two networks of agents have opposing objectives with regards to 
the optimization of a common objective function. In the resulting zero-sum game, individual agents collaborate with neighbors 
in their respective network and have only partial knowledge of the state of the agents in the other network. For the case 
when the interaction topology of each network is undirected, we synthesize a distributed saddle-point strategy and establish 
its convergence to the Nash equilibrium for the class of strictly concave-convex and locally Lipschitz objective functions. We 
also show that this dynamics does not converge in general if the topologies are directed. This justifies the introduction, in the 
directed case, of a generalization of this distributed dynamics which we show converges to the Nash equilibrium for the class 
of strictly concave-convex differentiable functions with globally Lipschitz gradients. The technical approach combines tools 
from algebraic graph theory, nonsmooth analysis, set-valued dynamical systems, and game theory. 
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1 Introduction 

Recent years have seen an increasing interest on net- 
worked strategic scenarios where agents may cooperate 
or compete with each other towards the achievement of 
some objective, interact across different layers, have ac- 
cess to hmited information, and are subject to evolving 
interaction topologies. This paper is a contribution to 
this body of work. Specifically, we consider a class of 
strategic scenarios in which two networks of agents are 
involved in a zero-sum game. We assume that the objec- 
tive function can be decomposed as a sum of concave- 
convex functions and that the networks have opposing 
objectives regarding its optimization. Agents collaborate 
with the neighbors in their own network and have partial 
information about the state of the agents in the other 
network. Such scenarios are challenging because infor- 
mation is spread across the agents and possibly multiple 
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layers, and networks, by themselves, are not the decision 
makers. Our aim is to design a distributed coordination 
algorithm that can be used by the agents to converge 
to the Nash equilibrium. Note that, for a 2-player zero- 
sum game of the type considered here, a pure Nash equi- 
librium corresponds to a saddle point of the objective 
function. 

Literature review. Multiple scenarios involving net- 
worked systems and intelligent adversaries in sensor 
netwo rks, filtering, fi n ance, and wireless communica - 
tions (|Kim and Bovdl 120081: IWan and Lemmonl . I2009D 
can be cast into the strategic framework described above. 
In such scenarios, the network objective arises as a result 
of the aggregation of agent-to-agent adversarial interac- 
tions regarding a common goal, and information is nat- 
urally distributed among the agents. The present work 
has connections with the literature on distributed opti- 
mization and zero-sum games. The distributed optimiza- 
tion of a sum of convex functions has been intensively 
studied in recent years, see e.g. (iNedic and Ozdaglai , 
2009; Wan and Lemmon, ' 2009t [Johansson et all . 12009 ; 
Zhu and_Martmez, 2012). These works bu il d on 

consensus-based dynamics do ifati-Sabcr ct al.l. 2007t 

Ren and Be"a7dl.l2008t[Bullo et al..,2009;.Mesb'ahi and Egerstedti 



20101) to find the solutions of the optimization problem 
in a variety of scenarios and are designed in discrete 
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time. Exceptions include ()Wang and Elial . l2010l 1201 iD 
on continuous-time di stributed optimization on undi- 
rected networks and (jGharesifard and Cortesl l2012cD 
on directed networks. 

Rega r ding zero-sum gam e s, the works (I Arrow et al.l , 
llQSl iMaistroskii HoTTL iNedic and Ozdealail l200l) 
study the convergence of discrete-time subgradi- 
ent dynamics to a saddle point. Continuous-time 
best-response dynamics for zero-sum games con- 
verges to the set of Nash equilibria for both convex- 
concave (iHofbauer and So rin. 2006) and quasiconvex- 
quasiconcave (jBarron ~al.l . 2010) functions. Under 
strict convexity-concavity assumptions, continuous- 
time s ubgradient flow dynamics converges to a saddle 
point ([Arrow et al.l . Il95ll Il958[ ). Asymptotic conver- 
gence is also guaranteed when the Hessian of the objec- 
tive function is positive definite in one argument and 
the function is linear in t he other (jArrow et al.L I1958L 
iFeiier and Paganinii |2010[ ). The distributed computa- 
tion of Nash equilibria in noncooperative games, where 
all players are adversarial, has been inve stigated under 
differ ent assumptions. The algorithm in (jLi and Ba^aii 
|1987[ ) relies on all-to-all communication and does not 
require players to know each other's p ayoff functions 
(which must be strongly c onvex) . In ([Frihauf et al.l . 
120121 : IStankovic ef all. |2012[ ). players are unaware of 
their own payoff functions but have access to the pay- 
off value of an action once it has been executed. These 
works design distributed strategies based on extremum 
seeking techniques to seek the set of Nash equilibria. 
Statement of contributions. We introduce the prob- 
lem of distributed convergence to Nash equilibria for 
two networks engaged in a strategic scenario. The net- 
works aim to either maximize or minimize a common 
objective function which can be written as a sum of 
concave-convex functions. Individual agents collaborate 
with neighbors in their respective network and have 
partial knowledge of the state of the agents in the other 
one. Our first contribution is the introduction of an 
aggregate objective function for each network which 
depends on the interaction topology through its Lapla- 
cian and the characterization of a family of points with 
a saddle property for the pair of functions. We show 
the correspondence between these points and the Nash 
equilibria of the overall game. When the graphs describ- 
ing the interaction topologies within each network are 
undirected, the gradients of these aggregate objective 
functions are distributed. Building on this observation, 
our second contribution is the synthesis of a consensus- 
based saddle-point strategy for adversarial networks 
with undirected topologies. We show that the proposed 
dynamics is guaranteed to asymptotically converge to 
the Nash equilibrium for the class of strictly concave- 
convex and locally Lipschitz objective functions. Our 
third contribution focuses on the directed case. We show 
that the transcription of the saddle-point dynamics to 
directed topologies fails to converge in general. This 
leads us to propose a generalization of the dynamics, 



for strongly connected weight-balanced topologies, that 
incorporates a design parameter. We show that, by ap- 
propriately choosing this parameter, the new dynamics 
asymptotically converges to the Nash equilibrium for 
the class of strictly concave-convex differentiable ob- 
jective functions with globally Lipschitz gradients. The 
technical approach employs notions and results from 
algebraic graph theory, nonsmooth and convex anal- 
ysis, set- valued dynamical systems, and game theory. 
As an intermediate result in our proof strategy for the 
directed case, we provide a generalization of the known 
characterization of cocoercivity of concave functions to 
concave-convex functions. 

The results of this paper can be understood as a general- 
iza tion to competing ne tworks of the results we obtained 
in (iGharesifard and Co rtes. 2012g) for distributed opti- 
mization. This generalization is nontrivial because the 
payoff functions associated to individual agents now also 
depend on information obtained from the opposing net- 
work. This feature gives rise to a hierarchy of saddle- 
point dynamics whose analysis is technically challeng- 
ing and requires, among other things, a reformulation of 
the problem as a constrained zero-sum game, a careful 
understanding of the coupling between the dynamics of 
both networks, and the generalization of the notion of 
cocoercivity to concave-convex functions. 
Organization. Section 2 contains preliminaries on non- 
smooth analysis, set- valued dynamical systems, graph 
theory, and game theory. In Section 3, we introduce the 
zero-sum game for two adversarial networks involved 
in a strategic scenario and introduce two novel aggre- 
gate objective functions. Section 4 presents our algo- 
rithm design and analysis for distributed convergence to 
Nash equilibrium when the network topologies are undi- 
rected. Section 5 presents our treatment for the directed 
case. Section 6 gathers our conclusions and ideas for fu- 
ture work. Appendix A contains the generalization to 
concave-convex functions of the characterization of co- 
coercivity of concave functions. 



2 Preliminaries 

We start with some notational conventions. Let M, M>o, 
Z, Z>i denote the set of real, nonnegative real, integer, 
and positive integer numbers, respectively. We denote 
by II • II the Euclidean norm on M'', d G Z>i and also 
use the short-hand notation Id — (1, . • . , 1)"^ and 0^ = 
(O, . . . ,0)^ G K"*. We let 1^ denote the identity matrix 



fdxd 



For matrices A € R'^iX'^a ^nd B G 



di,d2, 61,62 G Z> 1 , we let A (g) i? denote their Kronecker 
product. The function / : Xi x X2 K, with Xi C M.'^^ , 
X2 C K"^^ closed and convex, is concave- convex if it is 
conc ave in its fir s t arg ument and convex in the second 
one ()Rockafellarj . 119971) . A point ( 2) G Xi X X2 is 
a saddle point of / if /(xi, Xj) < f{xl , X2) < /{xl, X2) 
for all xi G Xi and X2 G X2. Finally, a set- valued map 
/ : K'' =t K'' takes elements of M."^ to subsets of R"*. 
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2.1 Nonsmooth analysis 

We recall some notions from nonsmooth analy- 
sis ()Clarkel I1983D . A function / : R*^ ^ R is locally 
Lipschitz at a; S R'' if there exists a neighborhood U of 
X and Cx & R>o such that \ f{y) — f{z)\ < Cx\\y — z\\, 
for y,z ^ U. f is locally Lipschitz on R'' if it is locally 
Lipschitz at x for all x G M.'^ and globally Lipschitz on 
R'' if for all y, z € M'* there exists C € R>o such that 
\f{y) ~ f{^)\ ^ C^Wu ~ -^ll- Locally Lipschitz functions 
are difFcrcntiable almost everywhere. The generalized 
gradient of / is 

df{x) = coi lim V/(xfc) \ Xk ^ x, Xk i flj U s\ , 

where fi/ is the set of points where / fails to be differ- 
entiable and S is any set of measure zero. 
Lemma 2.1 (Continuity of the generalized gradi- 
ent map) : Let / : R'' — > M &e a locally Lipschitz func- 
tion atx eM.'^. Then the set-valued map 9/ : R'^ =t R'' 
is upper semicontinuous and locally hounded at x G 
and moreover, df{x) is nonempty, compact, and convex. 
For / : M'' X M'' -> M and z £ R'^, we let dxf{x, z) de- 
note the generalized gradient of x i— >■ f{x, z). Similarly, 
for X e R^, we let dzf{x, z) denote the generalized gra- 
dient of z I— )■ f{x, z). A point a; e R^ with e df{x) is 
a critical point of /. A function / : — )■ M is regular at 
a; S R if for all v the right directional derivative of 
/, in the direction of v, exists at x and coincides with the 
generalized directional derivative of / at x in t he direc- 
tion of V. We refer the reader to ([Clarkel |1983[ ) for defi- 
nitions of these notions . A convex an d locally Lipschitz 
function at x is regular (|Clarkel . ri983l Proposition 2.3.6). 
The notion of regularity plays an important role when 
considering sums of Lipschitz functions. 
Lemma 2.2 (Finite sum of locally Lipschitz func- 
tions): Let he locally Lipschitz at x G M''. Then 
d{Y.7=if)i^) ^ J2"=i9r{x), and equality holds if f 
is regular for i £ {1, . . . , n}. 

A locally Lipschitz and convex function / satisfies, for 
all x, x' G R'^ and ^ G df{x), the first-order condition of 
convexity, 

fix')- f{x)>^-{x' ^x). (1) 

2.2 Set-valued dynamical systems 

Here, we recall some background on set- valued dynami- 
cal systems following ICortesI (|2Q08f ) . A continuous-time 
set- valued dynamical system on X C R*^ is a differential 
inclusion 

x{t) e ^P(x(t)) (2) 

where t e R>o and : X C R'' ^ M'* is a set- valued 
map. A solution to this dynamical system is an abso- 
lutely continuous curve x : [0, T] — >■ X which satisfies (2) 
almost everywhere. The set of equilibria of (2) is denoted 
by Eq(*) = {xeX\Oe ^'(a;)}. 



Lemma 2.3 (Existence of solutions): For ^ : R'' ^ 

R'' upper semicontinuous with nonempty, compact, and 
convex values, there exists a solution to (2) from any 
initial condition. 

The LaSalle Invariance Principle for set- valued continuous- 
time systems is helpful to establish the asymptotic 
stability properties of systems of the form (2). A set 
C X is weakly positively invariant with respect to 
^ if for any x G W, there exists x G X such that 
X G 'I'(a:). The set W is strongly positively invariant 
with respect to \E' if \E'(a;) C W, for all x g W. Finally, 
the set-valued Lie derivative of a differentiable function 
F : M"* ^ R with respect to * at a; G R'' is defined by 
£^V{x) = {v ■ \/V{x) I V e *(a;)}. 
Theorem 2.4 (Set-valued LaSalle Invariance 
Principle): Let W d X be a strongly positively invariant 
under (2) and V : X ^ M. a continuously differentiable 
function. Suppose the evolutions of (2) are hounded and 
max£vi,F(x) < or C^V{x) = $, for all x € W. If 
S'l.v = {a; e X I G C^i,V{x)}, then any solution x{t), 
t G R>o, starting in W converges to the largest weakly 
positively invariant set M contained in S^iy H W . When 
M is a finite collection of points, then the limit of each 
solution equals one of them. 

2.3 Graph theory 

We present some basic notions fro m algebraic graph the- 
ory following the exposition in (|Bullo et al.l . I2009D . A 
directed graph, or simply digraph, is a pair Q = (V,£), 
where V is a finite set called the vertex set and £ C V x V 
is the edge set. A digraph is undirected if (w, u) G £ any- 
time (u, v) G S. We refer to an undirected digraph as a 
graph. A path is an ordered sequence of vertices such that 
any ordered pair of vertices appearing consecutively is 
an edge of the digraph. A digraph is strongly connected if 
there is a path between any pair of distinct vertices. For 
a graph, we refer to this notion simply as connected. A 
weighted digraph is a triplet Q = (V,f , A), where (V,f) 
is a digraph and A G R"q" is the adjacency matrix of Q, 
with the property that Oij > if {vi,Vj) G £ and = 0, 
otherwise. The weighted out-degree and in-degree of Vi, 
? G {1, . . . , n}, are respectively, d^^t{vi) = J2]j=i o-ij and 
^in(^i) = Sj=i ^ji - The weighted out-degree matrix Dout 
is the diagonal matrix defined by (Dout)ii = '^out(*): foi" 
all i G {1, . . . , n}. The Laplacian matrix is L = Dout — A. 
Note that Ll„ = 0. If ^ is strongly connected, then zero 
is a simple eigenvalue of L. C/ is undirected if L = and 
weight-balanced if d^^t(w) = d^^{v), for all w G V. Equiv- 
alently, G is weight-balanced if and only if l^L — if and 
only if L -|- L"^ is positive semidefinite. Furthermore, if G 
is weight-balanced and strongly connected, then zero is 
a simple eigenvalue of L -F L"^. Note that any undirected 
graph is weight-balanced. 

2.4 Zero-sum games 

We recall basic game-theoretic notions following lBa^ar and Olsdeil 
(|1999i) . An n-player game is a triplet G = {P,X,U), 
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where P is the set of players with \P\ = n E Z>2, 
X = Xi X . . . X X„, Xi C M''' is the set of (pure) strate- 
gies of player vt G P, di G Z>i, and U — {ui, . . . ,Un), 
where Ui : X — >■ K is the payoff function of player Vi, 
i S {1, . . . , n}. The game G is called a zero-sum game 
if Yl^=i "^i ~ 0- outcome a;* € X is a (pure) Nash 
equilibrium of G if for alH G {1, . . . , n} and all Xi £ X^, 

where X-i denotes the actions of all players other than Vi . 
In this paper, we focus on a class of two-player zero-sum 
games which have at least one pure Nash equilibrium as 
the next result states. 

Theorem 2.5 (Minmax theorem): Let Xi C 

and X2 C , di,d2 G Z>i, be nonempty, compact, and 
convex. // m : Xi x X2 — > M is continuous and the sets 
{x' G Xi I u{x\y) > a} and {y' G X2 | u{x,y') < a} are 
convex for all x G Xi, y G X2, and a G R, then 

maxminu(a:, y) = minmax it (a;, y). 

X y y X 

Theorem 2.5 implies that the game G = ({wi, 112}, Xi x 
X2, (u, —u)) has a pure Nash equilibrium. 

3 Problem statement 

Consider two networks Ei and S2 composed of agents 
{wi, . . . , } and agents {wi, . . . , tUn^li respectively. 
Throughout this paper, Ei and E2 are either connected 
undirected graphs, c.f. Section 4, or strongly connected 
weight-balanced digraphs, c.f. Section 5. Since the lat- 
ter case includes the first one, throughout this section, 
we assume the latter. The state of Ei, denoted xi, be- 
longs to Xi C M'^S di G Z>i. Likewise, the state of E2, 
denoted X2^ belongs to X2 C K*^^, d2 G Z>i. In this pa- 
per, we do not get into the details of what these states 
represent (as a particular case, the network state could 
correspond to the collection of the states of agents in 
it). In addition, each agent Vi in Ei has an estimate 
x\ G K'*! of what the network state is, which may differ 
from the actual value xi . Similarly, each agent Wj in E2 

has an estimate X2 G M''^ of what the network state is. 
Within each network, neighboring agents can share their 
estimates. Networks can also obtain information about 
each other. This is modeled by means of a bipartite 
directed graph Ecng, called engagement graph, with dis- 
joint vertex sets {wi, . . . , w„j } and {wi, . . . , Wti2}i where 
every agent has at least one out-neighbor. According to 
this model, an agent in Ei obtains information from its 
out-neighbors in Eeng about their estimates of the state 
of E2, and vice versa. 
Figure 1 illustrates this concept. 

For each i G {1, . . . , ni}, let /{ : Xi x X2 — >■ R be a 
locally Lipschitz concave-convex function only available 
to agent Vi G Ei. Similarly, let : Xi x X2 R be a 
locally Lipschitz concave-convex function only available 
to agent Wj G E2, j G {1, . . . , 712}. The networks Ei and 




K2 



Fig. 1. Networks Ei and E2 engaged in a strategic scenario. 
Both networks are strongly connected and weight-balanced, 
with weights of 1 on each edge. Edges which correspond to 
Ecng are dashed. 

E2 are engaged in a zero-sum game with payoff function 
;/ : Xi X X2 R 

U{xi,X2) = ^f{{xi,X2) = ^/^(xi,X2), (3) 

where Ei wishes to maximize C/, while E2 wishes to min- 
imize it. The objective of the networks is therefore to 
settle upon a Nash equilibrium, i.e., to solve the follow- 
ing maxmin problem 

max min U{xi,X2)- (4) 

We refer to the this zero-sum game as the 2-network zero- 
sum game and denote it by Gadv-net = (Si, E2, Ecng, U). 
We assume that Xi C R**^ and X2 C R"^^ are compact 
convex. For convenience, let xi — (x}, . . . , a;"^)^ and 
^)-^ denote vector of agent estimates 
about the state of the respective networks. 
Remark 3.1 (Power allocation in communication 
channels in the presence of adversaries): Here 
we present an example frorn comm unications inspired 
by (jBovd and Vandenberghi. |2004 Section 5.5.3). Con- 
sider n Gaussian communication channels, each with 
signal power pi G R>o and noise power rji G R>o, for 
i G {1, . . . , n}. The capacity of each channel is propor- 
tional to log(l -I- (3pi/{ai -\- rji)), where (5 G R>o and 
(Ti > is the receiver noise. Note that capacity is con- 
cave in Pi and convex in r]i . Both signal and noise powers 
must satisfy a budget constraint, i.e., J2'i=iPi — P ^^'^ 
Y^i=i Vi — C', for some given P,C G R>o- Two networks 
of n agents are involved in this scenario, one, Ei, select- 
ing signal powers to maximize capacity, the other one, 
E2, selecting noise powers to minimize it. The network 
El has decided that mi channels will have signal power 
xi, while n — 1 — mi will have signal power a::2. The re- 
maining nth channel has its power determined to satisfy 
the budget constraint, i.e., P — mixi — (n — 1 — mi)x2- 
Likewise, the network E2 does something similar with 
TO2 channels with noise power yi, n — 1 — m2 channels 
with noise power y2, and one last channel with noise 
power C — TO2?/i — (n — 1 — m2)y2 ■ Each network is aware 
of the partition made by the other one. The individual 
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objective function of the two agents (one from Ei, the 
other from S2) making decisions on the power levels 
of the ith channel is the channel capacity itself. For 
i G {1, . . . , n — 1}, this takes the form 

r(x,2/) = iog(i + ^^), 

for some a, 6 € {l,2}.Herex = {x\,X2) andy = {yi,y2)- 
For i = n, it takes instead the form 



r{x,y) = log (1 + ^ 



/?(P — niiXi — (n 



m.i)x2) 



+ C- m2yi - (n - 1 - m2)y2 



Note that J2^=if^i^Ty) total capacity of the n 

communication channels. • 

3.1 Reformulation of the 2-network zero-sum game 

In this section, we describe how agents in each net- 
work use the information obtained from their neighbors 
to compute the value of their own objective functions. 
Based on these estimates, we introduce a reformulation 
of the Gadv-net = (^1,^2, Seng, U) which is instrumen- 
tal for establishing some of our results. 

Each agent in Ei has a locally Lipschitz, concave-convex 
function ff : R'^^ x M''2«2 _^ ^j^h the properties: 
(Extension of own payoff function) : for any xi G 



^\ X2 e 



® X2) = f{{xi,X2). 



(5a) 



(Distributed over Seng): there exists 



]^<i2|A/'|;^„g('Ui)| 

X2 e M'*="^ 



such that, for any x\ S 



fi{xi,X2)=^\{xi,Tr\{x2)), 



(5b) 



with n\ : R''2™2 ^d.2\N%ll^^(v,)\ ^^^^^ pi-ojcction of a;2 
to the values received by Vi from its out-neighbors 

in Seng. 

Equation (5a) states the fact that, when the estimates 
of all neighbors of an agent in the opponent's network 
agree, its evaluation should coincide with this estimate. 
Equation (5b) states the fact that agents can only use 
the information received from their neighbors in the in- 
teraction topology to compute their new estimates. 

Each agent in S2 has a function /| : M^^i"! x E'*^ ^ R 
with similar properties. The collective payoff functions 
of the two networks are 



Ux{Xx,X2) = ^fl{x\,X2), 
i=l 

"2 

U2{xi,X2) = ^/^(a;i,x^). 



(6a) 
(6b) 



In general, the functions Ui and U2 need not be the same. 
However, Ui{lni ® l„i ® X2) = t^2(lni ® Xi,lni (8) 
0:2), for any xi e X2 e M'^^ When both functions 
coincide, the next result shows that the original game 
can be lifted to a (constrained) zero-sum game. 
Lemma 3.2 (Reformulation of the 2-network 
zero-sum game): Assume thai the individual pay- 
off functions {/2}7=i satisfying (5) are such 
that the network payoff functions defined in (6) sat- 
isfy Ui = U2, and let U denote this common function. 
Then, the problem (4) on x W^^ is equivalent to the 



following problem on '. 



max min U{xi,X2), 



subject to Lixi = 0„idi, '^^2X2 = 0n2d2, (7) 
with he = L^(S) l<i^ and the Laplacian ofUg, £ G {1, 2}. 

Proof. The proof follows by noting that (i) C/(l„i 

a;i,l„2 (E) X2) — U{xi,X2) for all xi e K'^i and X2 G 
M.'^^ and (ii) since Qi and G2 are strongly connected, 
Liaji = 0„idi and L2a:2 = On^da iff xi = l„i (E) xi and 



X2 = Ir 



> X2 for some xi G R"^ and X2 G 



□ 



Remark 3.3 (Restrictions on extensions): The as- 
sumption of Lemma 3.2 does not hold in general for all 
sets of extensions satisfying (5a) and (5b). If the interac- 
tion topology is one-to-one (i.e., both networks have the 
same number of agents, the interaction topology is undi- 
rected, and each agent in the first network obtains infor- 
mation only from one agent in the opposing network), 
the natural extensions satisfy the assumption. Exam- 
ple 5.5 later provides yet another instance of a different 
nature. In general, determining if it is always possible to 
choose the extensions in such a way that the assumption 
holds is an open problem. • 

We denote by Gadv-net = (Si, S2, Seng, C^) the con- 
straincid zc!ro-sum game defined by (7) and refer to 
this situation by saying that Gadv-net can be lifted 
to Gadv-net- Our objcctive is to design a coordination 
algorithm that is implementable with the information 
that agents in Si and S2 possess and leads them to find 
a Nash equilibrium of Gadv-net, which corresponds to 
a Nash equilibrium of Gadv-nct by Lemma 3.2. Achiev- 
ing this goal, however, is nontrivial because individual 
agents, not networks themselves, are the decision mak- 
ers. From the point of view of agents in each network, 
the objective is to agree on the states of both their own 
network and the other network, and that the resulting 
states correspond to a Nash equilibrium of Gadv-net- 

The function U is locally Lipschitz and concave- 
convex. Moreover, from Lemma 2.2, the elements of 
dxiU{xi,X2) are of the form 



9{Xl,X2) — (5(x;,X2)' 



'9{x'^,x2)) ^ d^,U{xi,X2), 
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where 5^,3,^) ^ Sxi/i(4>^2), for i e {1, . . . ,ni}. Note 

that, because of (5b), we have dxifi{x\,ln2 X2) = 
dxifl{x\,X2)- A similar reasoning can be foUowed to de- 
scribe the elements of dx2U{xi,X2)- Next, we present a 
characterization of the Nash equilibria of Gadv-not, in- 
strumental for proving some of our upcoming results. 
Proposition 3.4 (Characterization of the Nash 
equilibria of Gadv-net): For Ei, S2 strongly connected 
and weight-balanced, define Fi and F2 by 

Fi{xi,Zi,X2) = -U{xi,X2) +X1L1Z1 + ^xfLiXi, 
F2{X2,Z2,Xi) = U{Xi,X2) + X2l'2Z2 + ]^X2^2X2- 

Then, Fi and F2 are convex in their first argument, lin- 
ear in their second one, and concave in their third one. 
Moreover, assume {x\, z\,X2, Z2) satisfies the following 
saddle property /or (Fi, ^2); (x\, 2;^) is a saddle point of 
{xi.zi) Fi{xi, Zi,X2) and {xl,Z2) is a saddle point 
of {X2,Z2) ^ F2{x2,Z2,xl). Then, 

(i) {xl,zl + Ini (81 ai,X2, z*2 -I- 1„2 02) satisfies the 
saddle property for {Fi , F2) for any ai € M'^^ , 02 S 
K'^^ and 

(a) {x\,X2) is a Nash equilibrium o/Gadv-net- 

Furthermore, 

(Hi) if {xl,X2) is a Nash equilibrium o/Gadv-net then 
there exists zl , Z2 such that {x^, zl,X2, Z2) satisfies 
the saddle property for {Fi , ^2) • 



4 Distributed convergence to Nash equilibria 
for undirected topologies 

In this section, we introduce a distributed dynamics 
which solves (7) when Ei and S2 are undirected. In par- 
ticular, we design gradient dynamics to find points with 
the saddle property for (^1,^2) prescribed by Proposi- 
tion 3.4. Consider the set-valued dynamics ^Nash-undir '■ 

xi +L1X1 +L1Z1 G 9xiC^(a;i,X2), (8a) 
ii = Lixi, (8b) 

±2 -\-'L2X2 + ^'2Z2 & -da:2U{Xi,X2), (8c) 

Z2 = L2X2, (8d) 

where xe,ze e M"'''s i e {1,2}. Note that (8a)-(8b) 
and (8c)-(8d) correspond to saddle-point dynamics of 
Fi in {xi, Zi) and F2 in {x2, Z2), respectively. Local so- 
lutions to this dynamics exist by virtue of Lemmas 2.1 
and 2.3. We characterize next its asymptotic conver- 
gence properties. 

Theorem 4.1 (Distributed convergence to Nash 
equilibria for undirected networks): Consider the 
zero-sum game Gadv-nct = (Ei, S2, Seng, C^), with Ei 
and S2 connected undirected graphs, Xi C R'^^ , X2 C 
compact and convex, and [/ : Xi x X2 — >■ M strictly 
concave- convex and locally Lipschitz. Assume Gadv-net 
can be lifted to Gadv-nct- Then, the projection onto the 
first and third com,ponents of the solutions of (8) asymp- 
totically converge to agreement on the Nash equilibrium 

of Gadv -net • 



Proof. The statement (i) is immediate. For (ii), since 
(xf , z* , a;2 , Z2 ) satisfies the saddle property, and the net- 
works are strongly connected and weight-balanced, we 
have x*i — lni®x\, x\ ^ W^^ , = 1,12 ® , ^2 G K''^ , 
Liz* e -da,fi{xl,x*2), and L2Z2 G da^^Uixl^xl). 
Thus there exist 3l,(a;*,a;j) ^ dx-,fl{xl,x*2), i G 

{l,...,ni}, and ,g2,(x^xj) ^ -dx^fiixl^xl), j G 
{1, . . . , 712}, such that 

^1^1 = (5^,(3,5; and 

^2Z2 = (5'2,(x*,x*)" • • • '5'2,(x5;,x*)))^- 

Noting that, for I G {1,2}, (ij, ® ld,)L<. = (1^^ ® 
\de){\-e IdJ = 1^,L \de = Odtxdmt, we deduce that 
Er=i5L(,.,,.) = Od, and E"=i52,(,.,,.) = 0d2, i.e., 
{xl,X2) is a Nash equilibrium. Finally for proving (iii), 
note that x^ = l„j (g) and X2 = 1^2 ® ^2- The result 
follows then from the fact that G dxiU{xl,X2) and 
G dx2U{xl,x'2) implies that there exists z^ G W''-'^^ 

and Z2 G R"^'^^ with Liz^ G dxiU{xl,X2) and L2Z2 ^ 

-dx2U{xi,x*2). □ 



Proof. Throughout this proof, since property (5b) 
holds, without loss of generality and for simplicity 
of notation, we assume that agcmts in Si have ac- 
cess to X2 and, similarly, agents in E2 have access 
to Xi. By Theorem 2.5, a solution to (6) exists. By 
the strict concavity-convexity properties, this solu- 
tion is, in fact, unique. Let us denote this solution by 
xl = lni<E)xl and 0:2 = ln2<S)X2- By Proposition 3. 4(iii), 
there exists z'l and Zj such that (xi, z^ , 033, Z2) G 
Eq('J'Nash-undir)- First, note that given any initial con- 
dition (a;;,z?,x^,z^) G (R"i'^i)2 x {W^^'^^ f, the set 

W^<>,4 = {{XuZi,X2,Z2) I 

Irfjz, = \d,)zl i G {1, 2}} (9) 

is strongly positively invariant under (8). Consider the 
function V : (M*^!"!)^ x (K''""")^ K>o defined by 

y(a;i,zi,a;2,Z2) 

+ ]^{X2 - X*2f{X2 - Xl) + \{Z2- zlY'{Z2 - Z^). 
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The function V is smooth. Next, we examine its 
set- valued Lie derivative along ^I/Nash-undir- Let ^ S 
''^*NaE.h-u„dir^(^i' ^ii ''^s, 2:2). By definition, there exists 
V e *Nash-undir(a;i, zi, X2, Z2), given by 



v={- Liaji - Li^i + 51,(0:1 ,a!2)) 

- L2X2 - L2Z2 - 5f2,(a!i,sc2)5 I^l^li 1^23:2), 

where 5i,(a,i,a!2) ^ Qa;! C/(a;i, 0:2) and g2,(xi,a;2) ^ 
dx2U{xi,X2), such that 

^ = -y . vy(a;i, 211,032, 2:2) 
= (xi - xl)'^{-LiXi - Lizi +51,(3,^^^,)) 

+ {X2 - X2)^(-L2X2 - L2Z2 - g2,{xu^2)) 
+ (Zi - Zi)'^LiXi + (Z2 - Z2)^L2X2. 

Notethat-LiXi-LiZi+gi ,^2) € --9^iFi(a;i, zi, ^2), 
Lia;i G 5ziFi(£Ci,zi,a;2), -L2a;2 - L2Z2 - g2,{xt,x2) G 
-9a!2-p2(a;i,Z2,a;2), and L2X2 e ^^2^2(032, Z2, a3i). Us- 
ing the first-order convexity property of Fx and F2 in 
their first two arguments, one gets 

C < -Fi(x*,zi,X2) - Fi(a;i,zi,X2) + F2{xl,Z2,xi) 

- F2{x2,Z2,Xi) + Fi{xi,Zi,X2) - Fi{xi,zl,X2) 
+ F2{X2,Z2,Xi) - F2{X2,Z2,Xi). 

Expanding each term and using the fact that {xl,zl,X2,Z2) 
G Eq(\l/Nash-undir), we simplify this inequality as 

(, < -U{xl,X2) + U{xi,X2) - zlLiXi 
- -aJiLiaJi - Z2L2X2 - -a;2L2a;2. 

By rearranging, we thus have 

^ < -F2{x2,z^,xl) - Fi{xi,zl,x*2). 

Next, since ^2(0;^, z^ , a;2) + Fi{xl, Z2,xl) = 0, we have 
^ < Fi{xl,zl,x'^) - Fi(a;i,z^,a;^) 

+ F2{X2,Z2,XI) - F2{X2,Z'2,X*x), 

yielding that ^ < 0. As a result, 

max>C5,„^^j,_„„^,^y(xi,zi,a32,Z2) < 0. 

As a by-product, we conclude that the trajectories 
of (8) are bounded. By virtue of the set-valued version 

of the LaSalle Invariance Principle, cf. Theorem 2.4, 
any trajectory of (8) starting from an initial condition 
{xi, Zi,X2, Z2) converges to the largest positively invari- 
ant set M in n V-\< V{x°,zlx°,z'i)). 
Let (xi,zi,X2,Z2) e M. Because M c 'S'*NaBh-u„dir,v, 



then Fi{xl,zl,xl) - Fi{xi,zl,xl) = 0, i.e., 

-U{xl,X2) + Lf{xi,X2) - xjLizl - ^x'^LiXi = 0. 

(10) 

Define now Gi : ]R"i^i x E"i'^i x M"2d2 _> R by 
Gi(a;i,zi,a;2) = i^i(a;i, zi, 032) - |a3fLia3i. Gi is con- 
vex in its first argument and linear in its second. Further- 
more, for fixed 032, the map (a;i,Zi) n> Gi(cci, Zi, a;2) 
has the same saddle points as (a3i, Zi) i^i(a;i, Zi, a;2). 
As a result, Gi{xl, zl,X2) — Gi{xi, zl,X2) < 0, or 
equivalently, -ij{xl,X2) + U{xi,X2) - xjLizl < 0. 
Combining this with (10), we have that LiXi = 
and —U{xl,x'2) + U{xi,x'2) = 0. Since U is strictly 
concave in its first argument a3i = x^. A similar ar- 
gument establishes that 032 = 032- Using now the fact 
that M is weakly positively invariant, one can deduce 
that heZi e — 5a;fC/(a3i,a32), for £ e {1,2}, and thus 

(a3l, Zl, 032, Z2) e Eq(*Nash-undir)- □ 

5 Distributed convergence to Nash equilibria 
for directed topologies 

Interestingly, the saddle-point dynamics (8) fails to con- 
verge when transcribed to the directed network setting. 
This observation is a consequence of the following result, 
which studies the stability of the linearization of the dy- 
namics (8), when the payoff functions have no contribu- 
tion to the linear part. 

Lemma 5.1 (Necessary condition for the conver- 
gence of (8) on digraphs): Let T,i he strongly con- 
nected and fi = 0, i e {1, . . . . rif] , for (. e {1,2}. Then, 
the set of network agreement configurations Sagree = 
{(l„i ® xi, 1„, ® Zl, 1„2 ® X2, 1„2 ® Z2) e (IR"i'^i)2 X 
(R"2'i2)2 I ^^^^^ e M'i'!,^ e {1,2}}, is stable under (8) 
iff, for any nonzero eigenvalue A of the Laplacian Li, 
I e {1,2}, one has \/3|Im(A)| < Re(A). 

Proof. In this case, (8) is linear with matrix 



'(-/V)®Li 



(11) 



and has iSagrec as equilibria. The eigenvalues of (11) are 
of the form \i , with A^ an eigenvalue of L^, for 

£ G {1,2} (since the eigenvalues of a Kronecker product 
are the product of the eigenvalues of the corresponding 
matrices). Since L^ = (g) 1^^, each eigenvalue of is 
an eigenvalue of L^. The result follows by noting that 
Re(A4^ ± ^i)) = i(TV3Im(A^) - Re(A^)). □ 

It is not difficult to construct examples of strictly 
concave-convex functions that have zero contribu- 
tion to the linearization of (8) around the solution. 
Therefore, such systems cannot be convergent if they 
fail the necessary condition identified in Lemma 5.1. 
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The counterexample provi d ed in our recent pa- 
per (|Gharesifard and Cortesl l2012c[ ) of strongly con- 
nected, weight-balanced digraphs that do not meet the 
stability criterium of Lemma 5.1 is therefore valid in 
this context too. 

From here on, we assume that the payoff functions are 
differentiable. We elaborate on the reasons for this as- 
sumption in Remark 5.3 later. Motivated by the obser- 
vation made in Lemma 5.1, we introduce a parameter 
a £ M>o in the dynamics of (8) as 

Xi + ahiXi + LiZi = WU{xi,X2), (12a) 
ii=Lia;i, (12b) 

±2 + a'L2X2 +'L2SZ2 ^ -VUixi,X2), (12c) 
Z2 = ^2X2- (12d) 

We next show that a suitable choice of a makes the 
dynamics convergent to the Nash equilibrium. 
Theorem 5.2 (Distributed convergence to Nash 
equilibria for directed networks): Consider the zero- 
sum game Gadv-nct = (Si, S2, Song, U), with Ei and S2 
strongly connected and weight-balanced digraphs, Xi C 
M.'^^ , X2 C M.'^^ compact and convex, and f7 : Xi x X2 — > 
R strictly concave-convex and differentiable with glob- 
ally Lipschitz gradient. Assume Gadv-nct can be lifted to 
Gadv-net such that U IS differentiable and has a globally 
Lipschitz gradient. Define h : K>o — >■ K 6j/ 



Kr' 



(1 +r2)' 



(13) 



where A™'" = min^^i ^jA* (L^ -I- Lj)}, A*(-) denotes the 
smallest non-zero eigenvalue and K £ R>o is the Lip- 
schitz constant of the gradient of U . Then there exists 
P* £ R>o with hip*) = such that for all < P < p* , 
the projection onto the first and third components of the 

solutions of (12) with a = asymptotically converge 

to agreement on the Nash eguilibrium of Gadv-not • 



Proof. Similarly to the proof of Theorem 4.1, we as- 
sume, without loss of generality, that agents in Ei 
have access to X2 and agents in E2 to Xi. For conve- 
nience, we denote the dynamics described in (12) by 

^-Nash-dir : (M''!"!)^ X (K''""=^)^ ^ (R''i"i)2 X (M''2n2)2^ 

Let (xi, z1,X2, Z2) be an arbitrary initial condition. 
Note that the set W^o ^0 defined by (9) is invariant un- 
der the evolutions of (12). By an argument similar to the 
one in the proof of Theorem 4.1, there exists a unique 
solution to (7), which we denote by 
and xl = 1„2 (8) ^2. By Proposition 3.4(i), there exists 
(xl,zl,x*2,z^) e Eq(^'Nash-dir) H W^o o Cousidcr the 



function V : (R'^i"i)2 x (R'^^"^)^ 



V{xi,Zi,X2,Z2) 



^{Xi-XD {Xi - Xl) + -{X2 - X*2) {X2-X*2) 

+ ^ivixuzi) - y(x*,z*))^(y(xi,zi) - y(x',2*)), 
+ 2 (^(^2,^2) ~ y(x*,z*))^(y(x2,z2) ^ y(cc',z*)), 



where 



Px( + zi, i G {1,2}, and P G 



M>o satisfies /S^ — aP + 2 — 0. This function is 
quadratic, hence smooth. Next, we consider ^ — 
-C^Na^h-dir^l^;!, ^1, 3^2, 2:2) given by 



^ = {-aLiXi - LiZi + VU{xi,X2), LiXi, -aL2X2 

- L2Z2 - ^U{xi,X2),l^2X2) ■ \7V{xi,Zi,X2,Z2) 

After some manipulation, one can show that 

e=i 
2 

+ J2 (- 1)'"' (^^ - f^^eU{xi, X2) 
i=l 
2 

where Ae, £ e {1, 2}, is 



A,, = 



-{a~P)Li -Li 



\i-Pia- P) + 1)U -PU) 
This equation can be written as 

2 ^ 

£=1 

2 

+ ^{xi - x*g,y(^^^^^^) - y(^^,,^,)f Ae{x*f,,y^^,^^,)) 
1=1 
2 

+ Y.^-^y~^{x, - x;f^^fi{xi,X2) 
1=1 

2 

+ Y^{-'^y~'^ P{y(^,,zi) -y(x|,z*))^V^,C/(a;i,a72), 
1=1 
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where Qe, £ G {1,2}, is given by 



Qe = (Lf + Li ) ( 



Note that, we have 



-/3> 



(14) 



= -{Va^,U{xl,X2), PVa^,U{xl,X2)), 
= {Va,,U{xi,X*2), pVa:2U{xi,X*2)). 

Thus, after substituting for y{xe,zz)-, we have 

(^ = ^-{xe- x}, Zi - zXfQ(,{xii - x\, Zi - z*i) 

1=1 ^ 

+ (1 + - xlf{^^Sj{x^,X2) - V^,U{xl,X2)) 

- (1 + P^){X2 - X*2f{Vx2U{Xi,X2) - Vx2U{Xi,X*2)) 
+ P{ZI - zlfiV^Mxi, X2) - Va>,U{xl X2)) 

- P{Z2 - Z*2f{Vx2U{xuX2) - Vx2U{xuX*2)), (15) 

where 



/ ^3_(iL±2)_t(_(l + t,2,\ 



where K G R>o is the Lipschitz constant of Vf7. We 
thus conclude that 



^1 



C < ^ |(a;£ - x},zt - z}YQi{xt - x},ze - z}] 

1=1 



\VxJj{xi,X2) - ^xiUixl,X2) 



+ ||Va,,f7(a;i,a;2) - Vx,f/(a;i,x^)||2) 

+ I3{zi - ztf{Vx,U{xi,X2) -Vx,U{xl,X2)) 

- /3(Z2 - Z^f{Vx2U{xi,X2) - Vx2U{xux*2)). 

One can write this inequaUty as displayed in (17), where 

X = {xi- x\,Zi - Z\,X2 - X*2,Z2 - Z*2, 
VxiU{xi,X2) - VxiU{xl,X2), 
Vx2U{Xl,X2) - Vx2U{xi,X2)). 

Since {xi,zi,X2,Z2) e W^o ^.o, we have (1^^ \de){ze - 

zl) = Od,,, ( € {li 2}, and hence it is enough to establish 
that Q is negative semidefinite on the subspace W = 

{{vi,V2,V3,V4^,V5,Ve) e (M"i''i)2 X (M"2'i2)2 X X 

R^^'^' I (Ij^ ® Idjw2 = Od,, (Ij^ ® ld,)«4 = 0„J. Using 
the fact that —^{1 + I3^)\nede is invertible, for £ e {1, 2}, 
we can express Q as 



Q 



N 



[Qi 
Q2 

-^{1 + P^)\n,d^ 

y 



-i(l + ^2)UdJ 



for £ e {1,2}. Each eigenvalue of Qi is of the form 



-iP" + 3/32 + 2) ± V(/3^ + 3/32 + 2)2 - 4/32 



2/3 



(16) 



where is an eigenvalue of + Lj, £ € {1, 2}. Using 
now Theorem A.l twice, one for {xi,X2), {xl,X2), and 
another one for {xi,X2), {xi, x^), we have 



{xi-xlf{V^,U{xi,X2)-^^,U{xl,X2)) < 

- ^ {\\Vx,UiXl,X2) - Vx^Uixl,X2)\\^) , 
-{X2 - X*2f{Vx^U{xi,X2) - Vx2U{xx,X*2)) < 

- ^ (||V,,C/(xi,X2) - Vx2U{xi,xm^) , 



where Qe = Qe + 



'0 



£€ {1,2}, and 



N 

















\ 













1 ^ 

1-1-^2 'nidi 




































1 , 

1-1-^2 In2a2 



















V 














'«2<i2 / 



Noting that W is invariant under N'^ (i.e., N'^W = W), 
all we need to check is that the matrix D is negative 
semidefinite on W. Clearly, 



-;^(i+r)ini<ii 




-^{l+0^)\„2d2 



9 



Qi,i2 

?1,21 Ql.22 

Q24I Q2,12 

Q2,21 Q2,22 

/3l„,d, 

V -/?i„,<j, 












K "nidi 















(17) 



(i+r) 



Q 



is negative definite. On the other hand, for £ e {1,2}, 
on (R"*''*)^, is an eigenvalue of Qi with multiphc- 
ity 2di and eigenspace generated by vectors of the form 
{\ni ® a, 0) and (0, 1^ ® b), with a, 6 e R''^ However, 
on {{vi,V2) eJR"^''0' I (1^, <^ \d,)v2 = OdJ, is an 
eigenvalue of Qi with multiplicity di and eigenspace gen- 
erated by vectors of the form (1„^, (g) a, 0). Moreover, on 
{(wi,w2) G (R"^'^0' I (In, <^ \d,)v2 = OdJ, the eigen- 
values of (]^§2y ( i„"<j^ ) are (i^^ with multiplicity 
•ni,di — di and wit h multiplicity nidi . Ther efore, us- 
ing Weyl's theorem (jHorn and Johnsonlll985l . Theorem 
4.3.7), we deduce that the nonzero eigenvalues of the 

sum Qi are upper bounded by A^,{Qi) + j^^rj- Thus, 
the eigenvalues of Q = ( '0^ cja ) ^'"'^ upper bounded by 
mm^^i ,2{A*iQi)} + (n§2)- From (16) and the defini- 
tion of h in (13), we conclude that the nonzero eigenval- 
ues of Q are upper bounded by /i(/3). It remains to show 
that there exists /3* S M>o with h{l3*) = such that 
for all < /3 < ,3* we have /i(/3) < 0. For r > small 
enough, h{r) < 0, since h{r) = -^A™'"r -I- 0{r^). Fur- 
thermore, \im.r^oo h{r) = K > 0. Hence, using the Mean 
Value Theorem, we deduce the existence of /?*. There- 
fore we conclude that £\i,^^^^^^.^V{xi, Zi,X2, Z2) < 0. 
As a by-product, the trajectories of (8) are bounded. 
Consequently, all assumptions of the LaSalle Invariance 
Principle, cf. Theorem 2.4, are satisfied. This result then 
implies that any trajectory of (8) starting from an ini- 
tial condition (a;^, zj, a;^, 2:2) converges to the largest 
invariant set M in S^^, ^ ^. i/ H W_o ^0. Note that if 

{xi,Zi,X2,Z2) e 'S'^Na.h-dir,^ n W^o ^o, thcu N'^ X £ 
kei{Q) X {0}. From the discussion above, we know ker((5) 
is generated by vectors of the form (l„j (g) oi, 0,0,0), 
(0,0, 1„2 (g) a2,0), ae e R'^^ j € {1,2}, and hence xi = 
x\ + ® ai, Zi = z^. Using the strict concavity- 
convexity, this then implies that xi = a;|. Finally, for 
(a;^, Zi, X2, Z2) G M, using the positive invariance of M, 
one deduces that {xl, Zi,X2, Z2) € Eq(\E'Nash-dir)- 1^ 

Remark 5.3 (Assumptions on payofT function): 

Two observations are in order regarding the assump- 
tions in Theorem 5.2 on the payoff function. First, the 



assumption that the payoff function has a globally Lip- 
schitz gradient is not too restrictive given that, since 
the state spaces are compact, standard boundedness 
conditions on the gradient imply the globally Lips- 
chitz condition. Second, we restrict our attention to 
differentiable payoff functions because locally Lipschitz 
functions with globally Lipsch itz generalized gradients 
are in fact differentiable, see (jGharesifard and Cortesl 
|2012d Proposition A.l). • 
Remark 5.4 (Comparison with best-response 
dynamics): Using the gradient flow has the advan- 
tage of avoiding the cumbersome computation of the 
best-response map. This, however, does not come for 
free. There are concave-convex functions for which the 
(distributed) gradient flow dynamics, unlike the best- 
response dynamics, fails to converge to the saddle point, 
see (Feijcr and Paganini, 2010) for an example. • 
We finish this section with an example. 
Example 5.5 (Distributed adversarial selection 
of signal and noise power via (12)): Recall the com- 
munication scenario described in Remark 3.1. Consider 
5 channels, {chi, ch2, cha, ch.4, chs}, for which the net- 
work Si has decided that {chi, ch^} have signal power 
Xl and {ch2,ch4} have signal power X2. Channel chs 
has its signal power determined to satisfy the budget 
constraint P e R>0 7 i-c, P ~ 2a;i — 2x2- Similarly, 
the network E2 has decided that chi has noise power 
j/i, {ch2, cha, ch4} have noise power j/2, and ch5 has 
noise power C — yi — 3j/2 to meet the budget con- 
straint C e M>o- We let x = {x^ , x"^ , x^ , x''' , x^) and 
y = {y'^,y^,y^,y'^,y'^), where x' = ix\,xi) G [0,P]2 
and y' = (yj , y|) G [0, C]^ for each i e {1, . . . , 5}. 
The networks Si and S2, which are weight-balanced and 
strongly connected, and the engagement topology Song 
are shown in Figure 2. Note that, according to this topol- 
ogy, each agent can observe the power employed by its 
adversary in its channel and, additionally, the agents in 
channel 2 can obtain information about the estimates 
of the opponent in channel 4 and vice versa. The payoff 
functions of the agents are given in Remark 3.1, where 
for simplicity we take ai = cri, for i G {1,3,5}, and 
(Ti = (72, for i G {2, 4}, with (Ti, (72 G K>o- 
This example fits into the approach described in Sec- 
tion 3.1 by considering the following extended payoff 
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Fig. 2. Networks Si, E2 and Ecng for the case study of Ex- 
ample 5.5. Edges which correspond to Eong are dashed. For 
i € {1, . . . , 5}, agents Vi and Wi are placed in channel chi. 

functions: 



1 . Pxl ^ 2 

3 ■ (72 + ' 3 



fl{x\y) ^t. iog(l + + ± log(l + -^), 



+ y2 



/f(x^z/)=log(l + -M^), 



/f (:^^ y) =^ log(l + + ^ log(l + ^^), 



1. Pxj 2 

(72 + yi^^3 



CT2 + ^2 



/f(x5,?/)=log(l 



(Ti+C-y5_3y5^ 



y^) =- log(l + + ^ log(l + -^), 



1 



CT2 + y2 



/^(o;, v^) =± iog(i + + t iog(i + 



CT2 + yj 



/l(a^,2/')=/i'(2^',y)- 



Note that these functions are strictly concave and thus 
the zero-sum game defined has a unique saddle point 
on the set [0, F]^ x [0, C]^. These functions satisfy (5) 
and Ui = C/2. Figure 3 shows the convergence of the 
dynamics (12) to the Nash equilibrium of the resulting 
2-network zero-sum game. • 

6 Conclusions and future work 

We have considered a class of strategic scenarios in which 
two networks of agents are involved in a zero-sum game. 
The networks aim to either maximize or minimize a 
common objective function. Individual agents collabo- 
rate with neighbors in their respective network and have 
partial knowledge of the state of the agents in the other 
one. We have introduced two aggregate objective func- 
tions, one per network, identified a family of points with 
a special saddle property for this pair of functions, and 
established their correspondence between the Nash equi- 
libria of the overall game. When the individual networks 
are undirected, we have proposed a distributed saddle- 
point dynamics that is implementable by each network 



via local interactions. We have shown that, for a class 
of strictly concave-convex and locally Lipschitz objec- 
tive functions, the proposed dynamics is guaranteed to 
converge to the Nash equilibrium. We have also shown 
that this saddle-point dynamics fails to converge for di- 
rected networks, even when they are strongly connected 
and weight-balanced. Motivated by this fact, we have in- 
troduced a generalization that incorporates a design pa- 
rameter. We have shown that this dynamics converges 
to the Nash equilibrium for strictly concave-convex and 
differentiable objective functions with globally Lipschitz 
gradients for appropriate parameter choices. An inter- 
esting venue of research is determining whether it is al- 
ways possible to choose the extensions of the individual 
payoff functions in such a way that the lifted objective 
functions coincide. Future work will also include relax- 
ing the assumptions of strict concavity-convexity and 
differentiability of the payoff functions, and the glob- 
ally Lipschitz condition on their gradients, extending 
our results to dynamic interaction topologies and non- 
zero sum games, and exploring the application to various 
areas, including collaborative resource allocation in the 
presence of adversaries, strategic social networks, collec- 
tive bargaining, and collaborative pursuit-evasion. 
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A Appendix 

The following result can be understood as a generaliza- 
tion of th e characterization of cocoercivi ty of concave 
functions (jColshtein and Tretvakovlll996l Lemma 6.7). 
Theorem A.l (Concave-convex difFerentiable 
functions with globally Lipschitz gradients): Let 
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/ : R'^i X R'^^ ~> R be a concave-convex differentiable 
function with globally Lipschitz gradient (with Lipschitz 
constant K e R^o)- For{x,y), {x' ,y') e W^^ x M''^ 

{x-x'f{VJ{x,y)-V,J{x\y')) 

+ iy-y'f{Vyf{x',y')^Vyf{x,y)) 

<-^{\\y.fix,y')-V.f{x',y')f 

+ ||V,/(x',y)-V,/(x',2/')f 
+ ||V,/(a;',2/)-V,/(x,y)f 

+ |lV,/(x,y')-V,/(x,y)|p). 

Proof. We start by noting that, for a concave function 
j : M'' — R with globahy Lipschitz gradient, the fol- 
lowin g inequality holds, see (,Golshtein and Tretvakovl 
119961 Equation 6.64), 

j(^)<J*-2^l|Vj(^)lP, (A.l) 

where j* — sup^gjjd j{x) and M is the Lipschitz constant 

of Vj. Given ix',y') e R"^^ x R'^^ define the map / : 
X R''^ R by 

fix, y) = /(x, y) - f{x', y)-{x- x'fV,f{x', y) 

+ fix, y) - fix, y') + iy- y'fWyfix, y'). 

Since the gradient of / is Lipschitz, the function / is 
differentiable almost everywhere. Thus, almost every- 
where, we have 

Vx/(a;, y) = Vj^fix, y) - V^^fix', y) + V^fix, y) 

- V^fix, y') -iy- y'f^^^yfix, y'), 
Vj,/(a;, y) = ^yfix, y) - Vyfix, y) + Vy/(x, y) 

- Vyfix, y') ^ix- x'f^y^xfix', y). 

In particular, note that W xfix' ,y') — Vj,/(x',y') — 0. 
Since a; i— ^ fix, y') and y i— ^ fix' , y) are concave and con- 
vex functions, respectively, we can use (A.l) to deduce 

fix, y') < -^WV.fix, y') - VJix', y')\\\ (A.2a) 
-fix',y)<^^\\Vyfix',y)-yyfix',y'W, (A.2b) 

where we have used the fact that sup^gRdi fix, y') = 
infj^gRda fix',y) = fix',y') = 0. Next, by definition of /, 

f{x, y') = fix, y') - fix, y') - (a; - xf\J.j,fix, y'), 
fix', y) = fix', y) - fix', y') - iy - y'fWyfix', y'). 



Using (A. 2), we deduce that 
fix,y')-fix',y) 

-ix^ x'fVxfix', y') + iy - y'fVyfix', y') 

<-^{\\^Jix,y')~^.fix',y')r 

+ \\Vyfix',y)~Vyfix',y')f). (A.3) 

The claim now follows by adding together (A.3) and 
the inequality that results by interchanging (x, y) and 
(x',y')in(A.3). □ 
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